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(57)Abstract: 

PROBLEM TO BE SOLVED: To extract a noticing area adjusted to human 
subjectively and to automatically device well-balanced composition. 
SOLUTION: A noticing area extracting device and an automatic 
composition deciding device 10 include an image forming device 14, which 
generates the original image of a panoramic image from video photographed 
by a camera 12. A noticing area extracting device 20 extracts a noticing 
area from the original image given from the device 14. Namely, evaluation 
matched with human subjectivity is given in accordance with the physical 
feature of the original image and the noticing area is extracted in 
accordance with the evaluated result. A composition cutting off device 22 
cuts out the extracted noticing area and an adjacent image area from the 
original image by referring to data on paintings painted by painters and 
photographs taken by photographers stored in a memory 24. Namely, the 
data can be cut out by the same composition as the case with painted 
images or photographed images. 
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Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] An attention field extractor equipped with an evaluation means to be the attention field extractor which 
extracts an attention field from a subject-copy image, and to evaluate the degree of [ ** ] based on the physical feature, 
and an extraction means to extract the aforementioned attention field according to the evaluation result of the 
aforementioned evaluation means. 

[Claim 2] The aforementioned physical feature is an attention field extractor containing the degree of different kind of a 
color according to claim 1 . 

[Claim 3] For the aforementioned evaluation means, the aforementioned physical feature is an attention field extractor 
according to claim 2 by which the degree of [**♦***] is evaluated based on at least one degree of different kind among 
the four degrees of different kind, including further the degree of different kind of the formal degree of different kind, 
and area, and the degree of different kind of a texture. 

[Claim 4] The aforementioned physical feature is an attention field extractor according to claim 1 to 3 which contains a 
color further. 

[Claim 5] The aforementioned physical feature is an attention field extractor according to claim 4 which contains further 
the area and spatial frequency of a field in the aforementioned subject-copy image. 

[Claim 6] The attention field extractor according to claim 1 to 5 further equipped with a photography means to photo a 
desired image, and a picture generation means to generate the aforementioned subject-copy image based on the 
aforementioned image. 

[Claim 7] The aforementioned picture generation means is an attention field extractor including a junction means to join 
the aforementioned image photoed with the aforementioned camera for every frame according to claim 6, including the 
driving means to which the aforementioned photography means carries out the rotation drive of a camera and the 
aforementioned camera. 

[Claim 8] Automatic composition determination equipment equipped with a maintenance means to hold the data about 
the criteria picture which is automatic composition determination equipment using the attention field extractor 
according to claim 1 to 7, and has criteria composition, and the cutoff means which cuts out the picture of an attention 
field from a subject-copy image with reference to the aforementioned criteria composition. 

[Claim 9] The aforementioned maintenance means is automatic composition determination equipment according to 
claim 8 which holds the data about two or more aforementioned criteria pictures, and is further equipped with a 
selection means to choose the aforementioned data which suited the picture of the aforementioned attention field. 
[Claim 10] The aforementioned data are automatic composition determination equipment according to claim 8 or 9 
which contains the image data corresponding to the aforementioned criteria picture, a photographic subjects 
configuration data, and a photographic subject's position data at least. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the attention field extractor cut out from a subject-copy image so that 
the composition of a picture that experts, such as a painter and a photographer, made the attention field which extracted 
and extracted the attention field may be suited and it may be especially settled, for example from a subject-copy image 
about the automatic composition determination equipment which used an attention field extractor and it, and the 
automatic composition determination equipment using it. 

[0002] Here, the field where people observe an attention field in a picture or an image is said. 
[0003] 

[Description of the Prior Art] As this kind of conventional technology, various methods are proposed about the 
technique to which a watcher extracts an attention field from a picture. (1) Milanese and Itti And Koch and others 
assumes the discontinuous part in a picture to be an attention field, and generates the feature map (shade picture) 
corresponding to two or more physical features acquired from a picture, next asks for the discontinuous part of each 
feature map, and is extracting as an attention field what unified them. 

[0004] (2) Milanese and others filtered the feature map using two or more Difference-of-ofiented-Gaussians filters with 
which the size was fixed in the discontinuous part of each feature map, and chose and searched for the filtering result 
from which an output serves as the maximum. Itti, Koch and others normalized each feature map with the square error 
with each average, and unified all the feature maps by linear combination. And the integrated feature map was 
recursively filtered with the Difference of Gaussian filter, and the local peak of the filtering result finally obtained was 
extracted as an attention field. 

[0005] Thus, Milanese, Itti and others were extracting the attention field by processing of pixel level, such as filtering 
and relaxation. 

[0006] (3) Martin, Takeuchi and others evaluated the brightness information obtained from a picture based on the 
information theory of Shannon, and made the attention field the portion with the high amount of information acquired as 
a result. By this method, the field, i.e., the field which looks complicated, where distribution is large and the bright field 
of a brightness value are mainly extracted. 

[0007] Moreover, with the conventional camera, human being had determined the composition of a photographic 

subject and a photographic subject manually. 

[0008] 

[Problem(s) to be Solved by the Invention] However, since the size of an attention field changed with pictures by the 
technique of of (1) and (2), it was difficult to extract an attention field exactly using the filter with which the size was 
fixed. Moreover, in the case of the picture which cannot not necessarily be said that the field which looks complicated, 
or a bright field is in agreement with an attention field like the black picture of a flower vase placed, for example in 
front of the wall of a complicated pattern by the technique of (3), it was difficult to extract an attention field exactly. 
Furthermore, by such proposal, there were few examples which performed collating with a watcher's (human being) 
subjectivity and an extraction result, and it was a question whether the attention field which actually suits human being's 
subjectivity can be extracted. 

[0009] Moreover, with the conventional camera, when ordinary men without the sense over a photograph determine 
composition, the good photograph of balance cannot necessarily be taken. 

[0010] So, the main purpose of this invention is offering the attention field extractor which can extract exactly the 
attention field which suited human being's subjectivity. 

[001 1] Moreover, other purposes of this invention are offering the automatic composition determination equipment 
which is automatic and can determine the good composition of balance. 
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[0012] 

[Means for Solving the Problem] The 1st invention is an attention field extractor which extracts an attention field from a 
subject-copy image, and is an attention field extractor equipped with an evaluation means to evaluate the degree of 
[ ** ] based on the physical feature (whenever for it to be conspicuous), and an extraction means to extract an attention 
field according to the evaluation result of an evaluation means. 

[0013] The 2nd invention is automatic composition determination equipment which used the attention field extractor of 
a publication for the 1st invention, and is automatic composition determination equipment equipped with a maintenance 
means to hold the data about the criteria picture which has criteria composition, and the cutoff means which cuts out the 
picture of an attention field from a subject-copy image with reference to criteria composition. 
[0014] 

[Function] An evaluation means evaluates the degree of [ ** ] by the attention field extractor of the 1st invention 
according to the physical feature of a subject-copy image. Here, the degree of [ ** ] means the parameter suitable for 
human being's subjectivity. An extraction means extracts a most conspicuous field from an evaluation result as an 
attention field. That is, since an evaluation means carries out evaluation which suited human being's subjectivity 
according to the physical feature, the attention field which suited human being's subjectivity can be extracted. 
[0015] For example, when the physical feature contains the degree of different kind of a color, the degree of [ ** ] can 
be evaluated based on the difference in the color of each field. 

[0016] Moreover, since it contains further the degree of different kind of the formal degree of different kind, and area, 
and the degree of different kind of a texture (pattern) whenever the color of the physical feature is heterogeneous in 
addition, if the degree of [ ** ] is evaluated based on at least one degree of different kind of these four degrees of 
different kind, according to the feature of a subject-copy image, the degree of [ ** ] can be evaluated exactly. 
[0017] Moreover, if it is the case where it evaluates also about three elements (a hue, saturation, lightness) of a color, 
the field near the conspicuous color (red) by human being's subjectivity can be estimated as a most conspicuous field. 
[0018] Furthermore, if it evaluates also about the area of each field in spatial frequency or a subject-copy image, most 
conspicuous evaluation of a field can be judged still more exactly. 

[0019] Moreover, the picture corresponding to the image which photoed the desired image with the photography means, 
for example, was photoed can be compounded, and a subject-copy image can also be generated. 
[0020] For example, if the position and height of a camera which are contained in a photography means are fixed, 360 
degrees rotates and it enables it to take a photograph, when a junction means joins the picture in every frame, the 
subject-copy image of a panorama picture is generable within the limits of 360 degrees. 

[0021] With the automatic composition determination equipment of the 2nd invention, a maintenance means makes a 
criteria picture the pictures picture and photograph corresponding to the photograph which the pictures which the 
painter drew, and the photographer took, and the data about this criteria picture are held. Since a cutoff means cuts out 
the picture of an attention field from a subject-copy image with reference to the data about this criteria picture, it is 
automatic and can determine the good composition of balance. 

[0022] Moreover, if the data about two or more criteria pictures are held, since a selection means can choose the 
composition suitable for the picture of an attention field, the good composition of balance can be determined about all 
pictures. 

[0023] Since they contain the image data corresponding to a criteria picture, a photographic subject's configuration data, 
and a photographic subject's position data at least, the data about the above criteria pictures choose the composition 
suitable for the attention field, are automatic and can determine the good composition of balance. That is, a photograph 
as if it photoed the photographic subject in the composition is generable. 
[0024] 

[Effect of the Invention] Since an attention field is extracted according to the evaluation result of the physical feature 
suitable for human being's subjectivity according to this invention, the attention field which suited human being's 
subjectivity can be extracted. 

[0025] According to other invention, since an attention field is cut out from a subject-copy image using the criteria 

composition of a criteria picture, it is automatic and the good composition of balance can be determined. 

[0026] The above-mentioned purpose of this invention, the other purposes, the feature, and an advantage will become 

still clearer from the detailed explanation of the following examples given with reference to a drawing. 

[0027] 

[Example] With reference to drawing 1 , the automatic composition determination equipment 10 of this example 
contains a video camera (only henceforth a "camera") 12. A photography position (a position and height) is fixed using 
a tripod etc., 360 degrees of cameras 12 rotate and they can be photoed. The image photoed with the camera 12 is 
inputted into picture generation equipment 14, and the panorama picture as a subject-copy image is generated from the 



Page3ofll 

photoed image. In addition, since 360 degrees of cameras 12 can be rotated, picture generation equipment 14 can 
generate a panorama picture within the limits of 360 degrees. 

[0028] Picture generation equipment 14 generates a panorama picture according to the flow view shown in drawing 2 
including a video capture 16 and picture junction equipment 18. That is, if photography is started with a camera 12, after 
picture generation equipment 14 starts processing, is Step SI, carries out the capture of the first video frame (the 1st 
frame) using a video capture 16 and generates a synthetic picture from the picture corresponding to oneth of them, it 
will generate a gray-scale picture from the synthetic picture using picture junction equipment 1 8. 
[0029] At continuing Step S3, the capture of the following frame is carried out and a gray-scale picture is generated like 
the 1st frame. At Step S5, in order to determine from which position the synthetic picture corresponding to the present 
frame (object frame) should be compounded, a search template is started from the gray-scale picture of an object frame 
as shown in drawing 3 . The width of face of this search template is 50 pixels, and the height is the same as a frame 
picture. Moreover, this search plate is cut down from the zero of an object frame. 

[0030] Then, at Step S7, the search range is set up from the gray- scale picture of a synthetic picture, and the picture of 
the same size as a search template is started from the search range. That is, the width of face of the search range is 100 
pixels, and the height is the same as a frame picture. Moreover, when the upper left of a frame picture is made into a 
zero (x y) (= (1 1)), the x-coordinate of a search plate is the position which subtracted only 100 from the width of face of 
a frame picture, and the y-coordinate is 1 . 

[0031] Next, at Step S9, from the search range, the picture of the same size as a search template is started, and the 
absolute value of the difference of the mutual pixel value corresponding to the picture and search plate which were cut 
down is computed. At continuing Step SI 1, it judges whether the difference of a pixel value is minimum (0). If it is 
"YES" at Step SI 1, the started picture will judge that it is the same size as a search plate, and will progress to Step SI 3. 
On the other hand, if it is "NO" at Step SI 1, after judging that the sizes of the picture and search plate which were cut 
down differ and shifting the search range of 1 pixel to the positive direction of a x axis at Step SI 5, it will return to Step 
S5. Thus, processing is repeated until the picture and search template which it started 1 pixel at a time with staggering 
become the same size. 

[0032] At Step SI 3, an X coordinate in case the absolute value of a difference is minimum (0) is computed. At 
continuing Step SI 7, a synthetic picture and an object frame picture are piled up by the computed X coordinate, and a 
new synthetic picture is generated. And at Step SI 9, it judges whether the gray-scale picture was generated from the 
generated synthetic picture, and the panorama picture was generated at Step S21 . If the panorama picture will not be 
generated if it is "NO" at Step S21 that is, it returns to Step S3. On the other hand, if it is "YES" at Step S21, it will 
judge that the panorama picture (subject-copy image) as shown in drawing 4 (A) was generated, a subject-copy image 
will be outputted to the latter attention field extractor 20 ( drawin g 1 ) at Step S23, and processing will be ended. 
[0033] Returning to drawing 1 , the attention field extractor 20 extracts a most conspicuous field (attention field) from 
the given subject-copy image. Specifically, the attention field extractor 20 processes according to the flow view shown 
in drawin g^ . That is, if a subject-copy image is given from picture generation equipment 14, the attention field 
extractor 20 will start processing and will carry out field division of the subject-copy image at Step S31. Specifically, as 
shown in drawin g4 (B), a subject-copy image is divided into a drawing field and a picture field, the method of this field 
division - 1997IEEE ~ setting - W.Y.Ma and B.S.Manjunath ** - the boundary detection method based on "edge 
flow" indicated to "Edge Flow: A Framework of Boundary Detection and Image Segmentation" is applied If it explains 
briefly, this method is edge flow which asks for the change direction of a color or a pattern in each place of a picture, 
and consists of strength (size) of the change direction and change. A vector is determined. And it is edge flow by 
repetitive operation. A vector is spread in the direction of each vector, and let the place where final vectors collide be 
the boundary line of each field. 

[0034] Therefore, the divided drawing field as shown in drawing 4 (C) is extracted, and Step S33 estimates the degree 

of a drawing field of [ ** ] at Step S3 5. That is, it asks for the degree parameter of a drawing field of [ ** ]. Here, the 

subjectivity evaluation experiment which the artificer conducted showed that the physical features required for 

evaluation of the degree of [ ** ] were the degree of different kind of the degree of different kind of the degree of 

different kind of a color, and a texture, and a form and the degree of different kind of area, a color, spatial frequency, 

and area. Moreover, generally, since the relation between human being's subjectivity evaluation result and a physical 

characteristic is expressed with a serpentine curve in many cases, an one ****** evening function is used for evaluation 

which is the degree of [ ** ] several 1 . 

[0035] 

[Equation 1] 

HOn. n) = Jih^(HT)dh 
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[0036] The performance index of the degree of [ ** ] as shown in several 2 is defined using this beta function 

[0037] 

[Equation 2] 

A, t = j;aSl-ar;)da, t 
a, = HET,+ FP U 
fc«U 

FP i!t : tMi0l*mtlc£{t«1HUIgff 

1 : SliSEi 
t : *M 

[0038] Moreover, the degree HET to the above-mentioned four degrees of different kind of each field shown in several 

2 of [ ** ] is defined by several 3. 
[0039] 

[Equation 3] 

RET, =wh 1 -HC i +wh t -HT, + tfh ! -HS,+wh 4 -HSh l 
fcfc'U 

met, : mmoim^ms. 
HC, : ®m<j)&<r>m&£. 
ht, : nmnftmnanm 

HS, : $HSi©S«<DHHJS 
HSh, : $ltti4>ftOM& 



[0040] Furthermore, the degree H of different kind of each physical feature is computed according to several 4, when 
standard deviation of dm and Difference d is set [ the difference of the feature value and the average feature value of all 
fields ] to std for the average of d and Difference d. 
[0041] 
[Equation 4] 

H(d.*.td)=|^-| 

[0042] That is, the degree HC of different kind of the color of each field is CIE. L* a* b* The result which computed 
and computed the standard deviation of the color difference of the average color of a field and the average color of all 
fields, a color difference average, and the color difference using the color difference formula in a consciousness equal 
color space is substituted for several 4, and it asks. In addition, the color difference formula is indicated in detail by 
1994 "a color science handbook." Specifically, it is the sexual desire news (R, G, B) of each pixel of a picture Equal 
perceived-color-space L* a* b* It changes and Euclidean distance in the space is made into the color difference. That is, 
a color difference formula is shown like several 5. 
[0043] 
[Equation 5] 

AEab*=[(AL*) , + (AaV+(Ab')] i 

[0044] Next, the degree HT of different kind of a texture is explained, a texture -- 1996IEEE -- setting -- B.S.Manjunath 
W.Y.Ma ** — it is expressed by the texture feature vector indicated to "Texture Features for Browsing and Retrieval of 
Image Data", and the difference between textures is expressed by the Euclidean distance between texture feature vectors 
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[0045] Here, a texture feature vector is expressed by the vector which uses as an element the response when filtering a 
picture by the Gabor filter bank constituted from two or more Gabor filters with which a size differs from a direction. 
However, since there is no orthogonality in each response when it filters with two or more Gabor filters, redundant 
information may be included in the filtered result. Then, the parameter of each filter in the Gabor filter bank is 
determined by the technique indicated by above-mentioned 1996IEEE. Specifically, it asks for theta, a, sigmau, and 
sigmav of the Gabor filter shown by several 6 by several 7. In addition, for this technique, the filter which adjoins as 
shown in drawing 6 is Half-Peak. In order to determine that the scale (size) and direction parameter of a filter touch and 
to express a textural facility, 24 filters (four scales, six directions) are used. 
[0046] 
[Equation 6] 



Mx,y)=a^G(x',y') 



G(x,y) = 



U=a' 



x'=xcos0+ysin0 
y'=-xsin0+ycos0 



cr,= 



27tCT u 



2ncr, 



tztzU 



Gmn(x.y) : 
m : 
n : 



0..S-1 
0..K-1 



S 
K 



[0047] 
[Equation 7] 
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— _ (a-1)U 



CT. 



■t«(*)[iMi-^j[art-aJijpff 



uh: r *' -«<itK ^©«x*/osas» 

[0048] A texture feature vector is expressed by this several 6. Therefore, the degree HT of different kind of the texture 
of each field is computed by substituting the average of distance with the mean vector of all fields, and distance, and the 
standard deviation of distance for several 4. 

[0049] Furthermore, the degree HS of different kind of the area of each field is computed by substituting the average of 
a difference with the average area of all fields, and a difference, and the standard deviation of a difference for several 4. 
[0050] In the degree of different kind of the hole which the degree of different kind of the appearance of a field and a 
field include, it unifies and the degree HSh of different kind of the form of each field is obtained further again, as shown 
in several 8. 
[0051] 
[Equation 8] 
HShi ,SHD^ 

fcffU 

HSh,: fttti®»®Aftg 
SHD,: mmi09U&O)R&& 
HOD,: ft&i<Dft<nm&& 

[0052] Here, it is necessary to take into consideration both the difference in the configuration itself, and the difference in 
the way of being visible depended on rotating a figure about the difference in a form. Then, a configuration can be 
described using the P type Fourier descriptor which Yoshinori Kamisaka indicated to "the new Fourier descriptor 
applicable also to an open curve" in 1984/3 "a telecommunication society paper magazine." In this case, if the power to 
each frequency after the P type Fourier transform is detected, power is in agreement when the configuration of two 
figures is the same. Moreover, if a Fourier descriptor is used, it restricts to the time when a configuration and an angle of 
rotation are in agreement, and a Fourier descriptor is in agreement. Therefore, a Fourier descriptor and the power to 
each frequency can estimate how many configurations and appearance of two figures are in agreement. 
[0053] As mentioned above, the feature of an appearance can be expressed by the vector which consists of a Fourier 
coefficient and power, and the difference of an appearance can be expressed using the Euclidean distance between these 
feature vectors. Therefore, the degree HSh of different kind of the appearance of each field is computed by substituting 
the average of distance with the mean vector of all fields, and distance, and the standard deviation of distance for several 
4. 

[0054] Next, the degree Hho of different kind of a hole is explained. In order to express the feature of the hole which a 
field includes, it is necessary to take into consideration not only the configuration of a hole but the number and hole site 
of a hole. Asking for the primary moment of a field can define the difference in a hole site. In addition, the primary 
moment is indicated by 1992 "an image-analysis handbook." Therefore, the feature of a hole was expressed by the 
vector as shown in several 9. In addition, turn of a hole is made into the order near the zero of a picture. 
[0055] 
[Equation 9] 
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ho = [nh, M, ha u , — , ha u . hb t . , — , hb*.. — , ha „„, , • • •, ha . hb*„ • • •, hb.« ] 







ho : 




nh ( : 




Mi : 




ha,.,., : 


i ©1 S § (DRO) j&fllHJiftft 


hbu.j : 


118* i Will g (DftO) j&OJn* 7- 



[0056] Therefore, the degree Hho of different kind of the hole of each field is computed by substituting the average of 
distance with the mean vector of all fields, and distance, and the standard deviation of distance for several 4. 
;0057] Moreover, the degree FP of [ ****** ] shown in several 2 can be defined as shown in several 10. 
;0058] 

Equation 10] 
fc«U 

FPu : mm\<Dftmtztsi't&#&mE$. 

PL : «tf i<Dgffljg«tt<DRB£ 

ps u : mmiommomss. 

wf , : 

[0059] In several 10, the degree PC of a color of [ ** ] is explained first. 

[0060] the conventional research ****-- 3 lightness in which the one where 2 saturation in which 1 warm color is 
conspicuous from cold color is higher is conspicuous is reported for the higher one to be conspicuous 1) is reported for 
red not to be dependent on a background color and to be a color with the high degree of [ ** ] by the experiment which 
**** and others conducted. Based on this result, in this example, it considers as the high hue whose red (0 R, G, B= 
255, 0) is the degree of [ ** ] most, and it is assumed that the degree of [ ** ] is high, so that a hue is close to red. HSI 
indicated in detail by above-mentioned 1992 "an image-analysis handbook" in the hue here - sugoroku - when it asks 
using a pyramid color model, a red (0 R, G, B= 255, 0) hue is 0 That is, the point which a HSI congruence hexagon- 
head drill color model makes BK (black) a zero, and carries out a counter electrode to it is set to W (white). And the 
straight line which connects BK and W is a center mostly, and the hexagon which makes a vertex R (red), M 
(MAZENDA), B (blue), C (cyanogen), G (green), and Y (yellow) so that it may intersect perpendicularly with the 
straight line is formed, thus, formed HSI ~ sugoroku - the color space of a pyramid color model is used In addition, a 
color space means the space of the rectangular three-dimensions system of coordinates used in order to display a 
perceived color as one point of space. That is, in this example, the degree helium of a hue of [ ** ] becomes large, so 
that a hue is close to 0, in order to search for a hue using a HSI congruence hexagon-head drill color model. For this 
reason, the degree helium of a hue of [ ** ] is computed by the formula shown in several 11. 
[0061] 

[Equation 11] 
He=1- ± 

fcf£U 

He: feffifi!3& 

«I L . h> n 0>m<& lih= k -h t T *) 

[0062] Moreover, about 2, an alignment-relation between saturation and the degree of [ ** ] is. therefore - this example 
— the degree of saturation of [ ** ] — HSI — sugoroku ~ we decided to use the saturation in a pyramid color model itself 

[0063] Furthermore, about 3, Semmelroth showed that the relation shown by several 12 was realized in the research on 
the relation between human being's amount of feelings, and lightness. In addition, the relation shown in several 12 is 
shown by Daisen, Imai, Wake and others in 1996 "a new edition feeling / consciousness psychology handbook." 
[0064] 
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[Equation 12] 

R=s a +kis-s,r 


S^S B 


R=s a -kis-s.r 


S<S b 






R : CXI 

Ss : *<D«SJt 







[0065] Moreover, in several 12, suiting well with human being's amount of feelings in the case of k= 0.65, m= 0.4, and 
n= 0.2 was shown. Therefore, the formula shown by Semmelroth was made into the degree of lightness of [ ** ] in this 
example. 

[0066] Linear combination of the degree to three elements (a hue, saturation, lightness) of the above colors of [ ** ] is 

carried out using several 13, and the degree PC of a color of [ ** ] is defined. 

[0067] 

[Equation 13] 
He+S+1 
3 

He: &ti(D$g& 
S : &S<DStBS 

[0068] Next, the degree PT of a texture (spatial frequency) of [ ** ] is explained. 

[0069] In human being's visual system, it is shown clearly that it has the property of a band-pass machine in which 
sensitivity serves as the maximum by specific spatial frequency. Until now, Kubota, Nishizawa and others are 
formulizing visual spatial frequency characteristics like several 14. In addition, the formula of spatial frequency 
characteristics is indicated by Kubota and others in 1986/5 "a telecommunication society paper magazine" at "the 3- 
dimensional weighting-of-noise function of a television system, and application to the high quality TV." In addition, the 
unit of spatial frequency is changed into cpd (the number of cycles per 1 degree of visual senses) in several 14. 
[0070] 

[Equation 14] 
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V = (X,Y)=A- 1- 



1+1 




1+ 



fc«U 



V : 


WHELK 


X : 




Y : 




A : 




d : 


&EJ£(lISA0>fegk) 
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[0071] The degree PT of a texture (spatial frequency) of [ ** ] is defined by the visual-sense reaction V shown in this 
several 14. 

[0072] Then, the degree PS of area of [ ** ] is explained. 

[0073] It is said that the direction of a near object is conspicuous from the center of a picture immediately after 
generally showing a picture, and if a point of regard is moved, the direction of the object near the point of regard which 
moved is said for the degree of [ ** ] to become high. This is defined as the degree of a place of [ ** ]. Thus, since the 
degree of each point of [ ** ] models a bird clapper small gradually whenever it keeps away from the point focusing on 
a certain point (point of regard), the two-dimensional gauss function shown in several 15 is used. However, it is 
assumed that the center of a gauss function changes according to the time which the point of regard moved. 
[0074] 

[Equation 15] 



[0075] Here, it depends for the breadth coefficient sigma of a gauss function on the distance d from a view to a picture. 
That is, if Distance d becomes large, the range which can be observed at once will become large. Generally, human 
being's visible visual field is 20 - 30 degrees. Then, the breadth coefficient sigma when setting the screen quantity of the 
picture shown to H, and making a viewing distance into d-H can be defined like several 16. In addition, Pix is the 
vertical number of pixels and theta is 20 - 30 degrees (0.176<tan theta / 2< 0.268). 
[0076] 

[Equation 16] 
a=2-Pix-d-tan -|- 

[0077] Here, if it takes into consideration that it is the set whose field is a pixel, the degree PS of the area of a field of 
[ ** ] can be expressed by the sum of the degree of the place of the pixel of a field of [ ** ]. Therefore, the degree PS of 
area of [ ** ] is defined like several 17. 
[0078] 

[Equation 17] 
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[0079] Thus, it asks for the drawing field where Step S3 7 estimates the degree of each drawing field of [ ** ], and the 
degree of [ ** ] serves as the maximum at Step S39 by the performance index of the defined degree of [ ** ]. That is, a 
most conspicuous field is determined as an attention field. Therefore, an attention field as shown in drawing 4 (D) can 
be extracted. 

[0080] In this example, in addition, each coefficient of the performance index of the degree of [ ** ] whl, wh2, wh3, 
[wh4] = [0.039, 0.010, 0.027, 0.020], and wfl, w£2 and = [0.132, 0.005, 0.100], and [wB] [m, n] = [1.358, 4.250] are 
used. Several 14 theta was made into 20 degrees, and the viewing distance d was set to lm, and the P type Fourier 
coefficient was carried out to to the 10th order. 

[0081] Moreover, although the degree of [ ** ] was evaluated using the eight above-mentioned physical features, this is 
for making two or more pictures which have all the features suit, and not necessarily needs to evaluate by this example 
about no physical features. 

[0082] Then, at Step S41, it asks for the drawing field contiguous to the determined attention field, the color difference 
of an attention field and an adjoining-sheets field and the Euclidean distance of a texture feature vector are found, that 
whose Euclidean distance of a texture feature vector the color difference is less than 2.0, and is less than 0.3 is extracted 
with an attention field, and processing is ended. In addition, the color difference is CIE as are mentioned above and 
shows several 5. L* a* b* It asks by the color difference formula in a consciousness equal color space. Moreover, a 
texture feature vector is called for according to several 6. 

[0083] It returns to drawin g 1 and the attention field extracted with the attention field extractor 20 is given to 
composition cutoff equipment 22. Memory 24 is connected to composition cutoff equipment 22, and two or more data 
related with the photograph which the picture which the painter drew, and the photographer took in memory 24 are 
memorized. Composition cutoff equipment 22 cuts out an attention field from a subject-copy image with reference to 
the data memorized by memory 24 according to the composition of a painter or a photographer. Specifically, 
composition cutoff equipment 22 cuts off an attention field according to the flow view shown in drawing 7 . That is, if 
an attention field and an adjoining drawing field are extracted by the attention field extractor 20, composition cutoff 
equipment 22 starts processing, will be Step S51 and will ask for the periphery pixel of an attention field. That is, it asks 
for the edge of the extracted attention field. At continuing Step S53, by the technique of the paper which above- 
mentioned Kamisaka indicated, it asks for the P type Fourier coefficient to the 10th order, and an attention field is made 
into a configuration vector. 

[0084] Here, the data memorized by memory 24 are the image data corresponding to criteria pictures, such as a pictures 
picture and a photograph, the data of the configuration vector which extracted the photographic subject from the 
pictures picture and photograph, and described a photographic subject's periphery configuration by the P type Fourier 
coefficient, and position data corresponding to the positional information of a photographic subject as shows drawing 8 . 
That is, a photographic subject's position data are data of the criteria composition of a criteria picture. The position of 
the zero (peak near the peak at the upper left of a picture) of b and a circumscription rectangle for the length of the side 
of a and length (wl, hi), [ the length of the side beside the circumscription rectangle surrounding a photographic 
subject ] It is expressed with a vector like several 18 when the position (position on the basis of the peak at the lower 
right of a picture) of the terminal point (peak near the peak at the lower right of a picture) of a circumscription rectangle 
is set to (w2, h2). 
[0085] 

[Equation 1 8] Position data [kxl , ky 1 , kx2, ky2] = [wl l\ 12 I h2 [ a and ]/b] [ a and hi ] [ b and w2 ] 
Then, the youg lid distance of the configuration vector searched for at Step S53 by Step S55 and the configuration 
vector of a pictures picture or a photograph stored in memory 24 is found, that is, matching with the data of criteria 
composition is performed, and the data with which the configuration of a field was most similar at Step S57 are 
acquired. In other words, the position data of the photographic subject of a pictures picture with the smallest Euclidean 
distance are acquired. That is, the optimal image data for an attention field and an adjoining drawing field is chosen 
from two or more image data. And at Step S59, it asks for the circumscription rectangle of an attention field, and some 
subject-copy images are cut out from the zero and terminal point of a circumscription rectangle according to a- 19 
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number. In addition, by several 19, * means multiplication. 
[0086] 

[Equation 19] Width of face of Xl=xl-W*kxl Yl=yl-H*kylX2=x2+W*kx2Y2=y2+W*ky2 ? however the 
circumscription rectangle of an attention field is set to W, and (xl, yl), and a terminal point are set [ height ] to (x2, y2) 
for the zero of H and a circumscription rectangle. Moreover, it considers as the height of a Y2= subject-copy image at 
the time of the width of face of X2= subject-copy image, and the height of a Y2> subject-copy image at the time of the 
width of face of Yl=l and X2> subject-copy image at the time of Xl=l and YK1 at the time of XK1. At this time, the 
peak coordinate of the rectangle (picture) to cut off is shown like several 20. 
[0087] 

[Equation 20] peak coordinate =[(X1, Yl) of the picture to cut off, (XI, Y2), and (X2, Yl) --] (X2, Y2) 
Then, the result (picture) cut off at Step S61 is outputted, and processing is ended. 

[0088] Therefore, according to the composition of a pictures picture as shown in drawing 9 (A), an attention field and an 
adjoining drawing field as shown in drawin g 9 (B) can be cut off. 

[0089] Since the field (attention field) which is most conspicuous from a subject-copy image according to the physical 
feature which was mentioned above is extracted according to this example, the attention field which suited human 
being's subjectivity can be extracted. 

[0090] Moreover, a photograph since the extracted attention field was cut off according to the composition of the 
photograph which the pictures which the painter drew, and the photographer took, as if it photoed the photographic 
subject in the composition is generable. That is, it is automatic and the good composition of balance can be determined. 
[0091] In addition, since this attention field extractor suits human being's subjectivity and can ask for the degree of a 
picture field of [ ** ], it is applicable to equipment which performs objective evaluation of each field of the picture used 
as the candidate for evaluation, or an image which is conspicuous and performs weighting to the field according to 
whenever (the degree of [ ** ]), for example in the quality evaluation of a digital compression picture and an image. 
[0092] Moreover, in a printing field, it is applicable to the equipment which is automatic and can judge the field which 
can make light of problems, such as a printing gap, and the field which is not so according to the degree of [ ** ] in 
automation of quality control. [0093] Furthermore, in the poster work used for a design field, especially an 
advertisement, it is applicable to the equipment by which it evaluates objective whether it is that the portion against 
which a company wants to appeal most is conspicuous. 

[0094] Moreover, if it controls rotation of a camera, a tilt, and zoom using an attention field extractor in being the dress 
with which a user is conspicuous, a camera is automatic and it can run after a user. A user's snapshot can be taken, if it 
follows, for example, a shutter is turned off by predetermined timing. Furthermore, using the photoed picture (subject- 
copy image) to composition determination equipment, it is automatic and the snapshot of the good user of balance can 
be created by determining composition. 
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(54) NOTICING AREA EXTRACTING DEVICE AND AUTOMATIC COMPOSITION DECIDING 
DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To extract a noticing area 
adjusted to human subjectively and to automatically 
device well-balanced composition. 
SOLUTION: A noticing area extracting device and an 
automatic composition deciding device 10 include an 
image forming device 14, which generates the original 
image of a panoramic image from video photographed by 
a camera 12. A noticing area extracting device 20 
extracts a noticing area from the original image given 
from the device 14. Namely, evaluation matched with 
human subjectivity is given in accordance with the 
physical feature of the original image and the noticing 
area is extracted in accordance with the evaluated 
result. A composition cutting off device 22 cuts out the 
extracted noticing area and an adjacent image area from 
the original image by referring to data on paintings 
painted by painters and photographs taken by 
photographers stored in a memory 24. Namely, the data 
can be cut out by the same composition as the case 
with painted images or photographed images. 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] An attention field extractor equipped with an evaluation means to be the attention field 
extractor which extracts an attention field from a subject-copy image, and to evaluate the 
degree of [ ** ] based on the physical feature, and an extraction means to extract the 
aforementioned attention field according to the evaluation result of the aforementioned 
evaluation means. 

[Claim 2] The aforementioned physical feature is an attention field extractor containing the 
degree of different kind of a color according to claim 1. 

[Claim 3] For the aforementioned evaluation means, the aforementioned physical feature is an 
attention field extractor according to claim 2 by which the degree of [ ****** ] is evaluated 
based on at least one degree of different, kind among the four degrees of different kind, including 
further the degree of different kind of the formal degree of different kind, and area, and the 
degree of different kind of a texture. 

[Claim 4] The aforementioned physical feature is an attention field extractor according to claim 1 
to 3 which contains a color further. 

[Claim 5] The aforementioned physical feature is an attention field extractor according to claim 4 
which contains further the area and spatial frequency of a field in the aforementioned subject- 
copy image. 

[Claim 6] The attention field extractor according to claim 1 to 5 further equipped with a 
photography means to photo a desired image, and a picture generation means to generate the 
aforementioned subject-copy image based on the aforementioned image. 

[Claim 7] The aforementioned picture generation means is an attention field extractor including a 
junction means to join the aforementioned image photoed with the aforementioned camera for 
every frame according to claim 6, including the driving means to which the aforementioned 
photography means carries out the rotation drive of a camera and the aforementioned camera. 
[Claim 8] Automatic composition determination equipment equipped with a maintenance means 
to hold the data about the criteria picture which is automatic composition determination 
equipment using the attention field extractor according to claim 1 to 7, and has criteria 
composition, and the cutoff means which cuts out the picture of an attention field from a 
subject-copy image with reference to the aforementioned criteria composition. 
[Claim 9] The aforementioned maintenance means is automatic composition determination 
equipment according to claim 8 which holds the data about two or more aforementioned criteria 
pictures, and is further equipped with a selection means to choose the aforementioned data 
which suited the picture of the aforementioned attention field. 

[Claim 10] The aforementioned data are automatic composition determination equipment 
according to claim 8 or 9 which contains the image data corresponding to the aforementioned 
criteria picture, a photographic subject's configuration data, and a photographic subject's position 
data at least. 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the attention field extractor cut out from a 
subject-copy image so that the composition of a picture that experts, such as a painter and a 
photographer, made the attention field which extracted and extracted the attention field may be 
suited and it may be especially settled, for example from a subject-copy image about the 
automatic composition determination equipment which used an attention field extractor and it, 
and the automatic composition determination equipment using it. 

[0002] Here, the field where people observe an attention field in a picture or an image is said. 
[0003] 

[Description of the Prior Art] As this kind of conventional technology, various methods are 
proposed about the technique to which a watcher extracts an attention field from a picture. (1) 
Milanese and Itti And Koch and others assumes the discontinuous part in a picture to be an 
attention field, and generates the feature map (shade picture) corresponding to two or more 
physical features acquired from a picture, next asks for the discontinuous part of each feature 
map, and is extracting as an attention field what unified them. 

[0004] (2) Milanese and others filtered the feature map using two or more Difference-of- 
ofiented-Gaussians filters with which the size was fixed in the discontinuous part of each feature 
map, and chose and searched for the filtering result from which an output serves as the 
maximum. Itti, Koch and others normalized each feature map with the square error with each 
average, and unified all the feature maps by linear combination. And the integrated feature map 
was recursively filtered with the Difference of Gaussian filter, and the local peak of the filtering 
result finally obtained was extracted as an attention field. 

[0005] Thus, Milanese, Itti and others were extracting the attention field by processing of pixel 
level, such as filtering and relaxation. 

[0006] (3) Martin, Takeuchi and others evaluated the brightness information obtained from a 
picture based on the information theory of Shannon, and made the attention field the portion 
with the high amount of information acquired as a result. By this method, the field, i.e., the field 
which looks complicated, where distribution is large and the bright field of a brightness value are 
mainly extracted. 

[0007] Moreover, with the conventional camera, human being had determined the composition of 

a photographic subject and a photographic subject manually. 

[0008] 

[Problem(s) to be Solved by the Invention] However, since the size of an attention field changed 
with pictures by the technique of of (1) and (2), it was difficult to extract an attention field 
exactly using the filter with which the size was fixed. Moreover, in the case of the picture which 
cannot not necessarily be said that the field which looks complicated, or a bright field is in 
agreement with an attention field like the black picture of a flower vase placed, for example in 
front of the wall of a complicated pattern by the technique of (3), it was difficult to extract an 
attention field exactly. Furthermore, by such proposal, there were few examples which performed 
collating with a watcher s (human being) subjectivity and an extraction result, and it was a 
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question whether the attention field which actually suits human being's subjectivity can be 
extracted. 

[0009] Moreover, with the conventional camera, when ordinary men without the sense over a 
photograph determine composition, the good photograph of balance cannot necessarily be taken. 
[0010] So, the main purpose of this invention is offering the attention field extractor which can 
extract exactly the attention field which suited human being's subjectivity. 
[0011] Moreover, other purposes of this invention are offering the automatic composition 
determination equipment which is automatic and can determine the good composition of balance. 

[0012] 

[Means for Solving the Problem] The 1st invention is an attention field extractor which extracts 
an attention field from a subject-copy image, and is an attention field extractor equipped with an 
evaluation means to evaluate the degree of [ ** ] based on the physical feature (whenever for it 
to be conspicuous), and an extraction means to extract an attention field according to the 
evaluation result of an evaluation means. 

[0013] The 2nd invention is automatic composition determination equipment which used the 
attention field extractor of a publication for the 1st invention, and is automatic composition 
determination equipment equipped with a maintenance means to hold the data about the criteria 
picture which has criteria composition, and the cutoff means which cuts out the picture of an 
attention field from a subject-copy image with reference to criteria composition. 
[0014] 

[Function] An evaluation means evaluates the degree of [ ** ] by the attention field extractor of 
the 1st invention according to the physical feature of a subject-copy image. Here, the degree of 
[ ** ] means the parameter suitable for human being's subjectivity. An extraction means extracts 
a most conspicuous field from an evaluation result as an attention field. That is, since an 
evaluation means carries out evaluation which suited human beings subjectivity according to the 
physical feature, the attention field which suited human being's subjectivity can be extracted. 
[0015] For example, when the physical feature contains the degree of different kind of a color, 
the degree of [ ** ] can be evaluated based on the difference in the color of each field. 
[0016] Moreover, since it contains further the degree of different kind of the formal degree of 
different kind, and area, and the degree of different kind of a texture (pattern) whenever the 
color of the physical feature is heterogeneous in addition, if the degree of [ ** ] is evaluated 
based on at least one degree of different kind of these four degrees of different kind, according 
to the feature of a subject-copy image, the degree of [ ** ] can be evaluated exactly. 
[0017] Moreover, if it is the case where it evaluates also about three elements (a hue, saturation, 
lightness) of a color, the field near the conspicuous color (red) by human being s subjectivity can 
be estimated as a most conspicuous field. 

[0018] Furthermore, if it evaluates also about the area of each field in spatial frequency or a 
subject-copy image, most conspicuous evaluation of a field can be judged still more exactly. 
[0019] Moreover, the picture corresponding to the image which photoed the desired image with 
the photography means, for example, was photoed can be compounded, and a subject-copy 
image can also be generated. 

[0020] For example, if the position and height of a camera which are contained in a photography 
means are fixed, 360 degrees rotates and it enables it to take a photograph, when a junction 
means joins the picture in every frame, the subject-copy image of a panorama picture is 
generable within the limits of 360 degrees. 

[0021] With the automatic composition determination equipment of the 2nd invention, a 
maintenance means makes a criteria picture the pictures picture and photograph corresponding 
to the photograph which the pictures which the painter drew, and the photographer took, and the 
data about this criteria picture are held. Since a cutoff means cuts out the picture of an 
attention field from a subject-copy image with reference to the data about this criteria picture, it 
is automatic and can determine the good composition of balance. 

[0022] Moreover, if the data about two or more criteria pictures are held, since a selection 
means can choose the composition suitable for the picture of an attention field, the good 
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composition of balance can be determined about all pictures. 

[0023] Since they contain the image data corresponding to a criteria picture, a photographic 
subject's configuration data, and a photographic subject's position data at least, the data about 
the above criteria pictures choose the composition suitable for the attention field, are automatic 
and can determine the good composition of balance. That is, a photograph as if it photoed the 
photographic subject in the composition is generable. 
[0024] 

[Effect of the Invention] Since an attention field is extracted according to the evaluation result 
of the physical feature suitable for human beings subjectivity according to this invention, the 
attention field which suited human beings subjectivity can be extracted. 
[0025] According to other invention, since an attention field is cut out from a subject-copy 
image using the criteria composition of a criteria picture, it is automatic and the good 
composition of balance can be determined. 

[0026] The above-mentioned purpose of this invention, the other purposes, the feature, and an 
advantage will become still clearer from the detailed explanation of the following examples given 
with reference to a drawing. 
[0027] 

[Example] With reference to drawin g 1 , the automatic composition determination equipment 10 
of this example contains a video camera (only henceforth a "camera") 12. A photography 
position (a position and height) is fixed using a tripod etc., 360 degrees of cameras 12 rotate and 
they can be photoed. The image photoed with the camera 12 is inputted into picture generation 
equipment 14, and the panorama picture as a subject-copy image is generated from the photoed 
image. In addition, since 360 degrees of cameras 12 can be rotated, picture generation equipment 
14 can generate a panorama picture within the limits of 360 degrees. 

[0028] Picture generation equipment 14 generates a panorama picture according to the flow view 
shown in dr awi ng 2 including a video capture 16 and picture junction equipment 18. That is, if 
photography is started with a camera 12, after picture generation equipment 14 starts 
processing, is Step SI, carries out the capture of the first video frame (the 1st frame) using a 
video capture 16 and generates a synthetic picture from the picture corresponding to oneth of 
them, it will generate a gray-scale picture from the synthetic picture using picture junction 
equipment 18. 

[0029] At continuing Step S3, the capture of the following frame is carried out and a gray-scale 
picture is generated like the 1 st frame. At Step S5, in order to determine from which position the 
synthetic picture corresponding to the present frame (object frame) should be compounded, a 
search template is started from the gray-scale picture of an object frame as shown in drawin g 
3 . The width of face of this search template is 50 pixels, and the height is the same as a frame 
picture. Moreover, this search plate is cut down from the zero of an object frame. 
[0030] Then, at Step S7, the search range is set up from the gray-scale picture of a synthetic 
picture, and the picture of the same size as a search template is started from the search range. 
That is, the width of face of the search range is 100 pixels, and the height is the same as a frame 
picture. Moreover, when the upper left of a frame picture is made into a zero (x y) (= (1 1)), the 
x-coordinate of a search plate is the position which subtracted only 100 from the width of face 
of a frame picture, and the y-coordinate is 1 . 

[0031] Next, in step S9, from the search range, the picture of the same size as a search 
template is started, and the absolute value of the difference of the mutual pixel value 
corresponding to the picture and search plate which were cut down is computed. At continuing 
Step S1 1, it judges whether the difference of a pixel value is minimum (0). If it is "YES" at Step 
S1 1, the started picture will judge that it is the same size as a search plate, and will progress to 
Step S13. On the other hand, if it is "NO" at Step S1 1, after judging that the sizes of the picture 
and search plate which were cut down differ and shifting the search range of 1 pixel to the 
positive direction of a x axis at Step S15, it will return to Step S5. Thus, processing is repeated 
until the picture and search template which it started 1 pixel at a time with staggering become 
the same size. 

[0032] At Step S13, an X coordinate in case the absolute value of a difference is .minimum (0) is 
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computed. At continuing Step S17, a synthetic picture and an object frame picture are piled up 
by the computed X coordinate, and a new synthetic picture is generated. And at Step S19, it 
judges whether the gray-scale picture was generated from the generated synthetic picture, and 
the panorama picture was generated at Step S21. If the panorama picture will not be generated if 
it is "NO" at Step S21 that is, it returns to Step S3. On the other hand, if it is "YES" at Step 
S21, it will judge that the panorama picture (subject-copy image) as shown in drawing 4 (A) was 
generated, a subject-copy image will be outputted to the latter attention field extractor 20 
( drawing 1 ) at Step S23, and processing will be ended. 

[0033] Returning to drawing 1 , the attention field extractor 20 extracts a most conspicuous field 
(attention field) from the given subject-copy image. Specifically, the attention field extractor 20 
processes according to the flow view shown in drawing 5 . That is, if a subject-copy image is 
given from picture generation equipment 14, the attention field extractor 20 will start processing 
and will carry out field division of the subject-copy image at Step S31. Specifically, as shown in 
draw ing 4 (B), a subject-copy image is divided into a drawing field and a picture field, the method 
of this field division — 1997IEEE — setting — W.Y.Ma and B.S.Manjunath ** — the boundary 
detection method based on "edge flow" indicated to "Edge Flow:A Framework of Boundary 
Detection and Image Segmentation" is applied If it explains briefly, this method is edge flow 
which asks for the change direction of a color or a pattern in each place of a picture, and 
consists of strength (size) of the change direction and change. A vector is determined. And it is 
edge flow by repetitive operation. A vector is spread in the direction of each vector, and let the 
place where final vectors collide be the boundary line of each field. 

[0034] Therefore, the divided drawing field as shown in drawing 4 (C) is extracted, and Step S33 
estimates the degree of a drawing field of [ ** ] at Step S35. That is, it asks for the degree 
parameter of a drawing field of [ ** ]. Here, the subjectivity evaluation experiment which the 
artificer conducted showed that the physical features required for evaluation of the degree of 
[ ** ] were the degree of different kind of the degree of different kind of the degree of different 
kind of a color, and a texture, and a form and the degree of different kind of area, a color, spatial 
frequency, and area. Moreover, generally, since the relation between human being's subjectivity 
evaluation result and a physical characteristic is expressed with a serpentine curve in many 
cases, an one ****** evening function is used for evaluation which is the degree of [ ** ] 
several 1. 
[0035] 
[Equation 1] 
H(m, n) = x;h^(1-h" 1 )dh 

[0036] The performance index of the degree of [ ** ] as shown in several 2 is defined using this 
beta function. 
[0037] 
[Equation 2] 

A, t =j";ar;a-ar;)da, t 

a s = HET, + FP it 
tzt£ L. 



i • $U3i 
t : B*[H] 
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[0038] Moreover, the degree HET to the above-mentioned four degrees of different kind of each 
field shown in several 2 of [ ** ] is defined by several 3. 
[0039] 
[Equation 3] 

HET, =wh,-HC i + wh,-HT,+wh 1 - HS, + wh.-HSh, 
fc*£U 

HET, : ttUi«>Jttttt«g& 

hc, : nmm&omnBL 
ht, : &m\o)mf*o>mn&. 
hs, : m\a>mn<r>mnm 
HSh, : ®®\o>m<D&mm. 

[0040] Furthermore, the degree H of different kind of each physical feature is computed 
according to several 4, when standard deviation of dm and Difference d is set [ the difference of 
the feature value and the average feature value of all fields ] to std for the average of d and 
Difference d. 
[0041] 
[Equation 4] 

H(d. dm.std) = |-^| 

[0042] That is, the degree HC of different kind of the color of each field is CIE. L* a* b* The 
result which computed and computed the standard deviation of the color difference of the 
average color of a field and the average color of all fields, a color difference average, and the 
color difference using the color difference formula in a consciousness equal color space is 
substituted for several 4, and it asks. In addition, the color difference formula is indicated in 
detail by 1994 "a color science handbook." Specifically, it is the sexual desire news (R, G, B) of 
each pixel of a picture Equal perceived-color-space L* a* b* It changes and Euclidean distance 
in the space is made into the color difference. That is, a color difference formula is shown like 
several 5. 
[0043] 
[Equation 5] 

A Eab* = [ ( A L')'+ ( A a Y+ ( A b*) ]* 

[0044] Next, the degree HT of different kind of a texture is explained, a texture — 1996IEEE — 
setting — B.S.Manjunath W.Y.Ma ** — it is expressed by the texture feature vector indicated to 
"Texture Features for Browsing and Retrieval of Image Data", and the difference between 
textures is expressed by the Euclidean distance between texture feature vectors 
[0045] Here, a texture feature vector is expressed by the vector which uses as an element the 
response when filtering a picture by the Gabor filter bank constituted from two or more Gabor 
filters with which a size differs from a direction. However, since there is no orthogonality in each 
response when it filters with two or more Gabor filters, redundant information may be included in 
the filtered result. Then, the parameter of each filter in the Gabor filter bank is determined by 
the technique indicated by above-mentioned 1996IEEE. Specifically, it asks for theta, a, sigmau. 
and sigmav of the Gabor filter shown by several 6 by several 7. In addition, for this technique, the 
filter which adjoins as shown in drawing 6 is Half-Peak. In order to determine that the scale 
(size) and direction parameter of a filter touch and to express a textural facility, 24 filters (four 
scales, six directions) are used. 
[0046] 
[Equation 6] 
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<L(x, y)=a-.G(x',y') 



G(x ' * )= fe^) 6XP [-t(-#I + ii))exp(27r jUx) 



U=a' 

x'=xcos0+ysin 8 
y'=-xsin0+ycos0 

]___ 

a '~ 2n<j* 

1 

fc«U 

Gmn(x, y) : XHHD=m, ;5<_.tD=ntf>B#<Dr **-JlM* 

ra : 0 S-1 

n : 0..K-1 

S : AHHfc 

K : 

[0047] 
[Equation 7] 

a = (U/U,) ,?T 

(a+1)77Trv2" 



Uh : r ** -*7<W^ >*a>»x*/D®£» 
Ul : r * # -»7«Mi\ # ^(OA/jMf 3 ^^^^ 

[0048] A texture feature vector is expressed by this several 6. Therefore, the degree HT of 
different kind of the texture of each field is computed by substituting the average of distance 
with the mean vector of all fields, and distance, and the standard deviation of distance for 
several 4. 

[0049] Furthermore, the degree HS of different kind of the area of each field is computed by 
substituting the average of a difference with the average area of all fields, and a difference, and 
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the standard deviation of a difference for several 4. 

[0050] In the degree of different kind of the hole which the degree of different kind of the 

appearance of a field and a field include, it unifies and the degree HSh of different kind of the 

form of each field is obtained further again, as shown in several 8. 

[0051] 

[Equation 8] 

H;h ■ SHDi +H0D, 

fcffL 

HSh, : mmxDBo&ns. 
shd,: ftwowmoRnm 
hod,: femvKamM 

[0052] Here, it is necessary to take into consideration both the difference in the configuration 
itself, and the difference in the way of being visible depended on rotating a figure about the 
difference in a form. Then, a configuration can be described using the P type Fourier descriptor 
which Yoshinori Kamisaka indicated to "the new Fourier descriptor applicable also to an open 
curve" in 1984/3 "a telecommunication society paper magazine." In this case, if the power to 
each frequency after the P type Fourier transform is detected, power is in agreement when the 
configuration of two figures is the same. Moreover, if a Fourier descriptor is used, it restricts to 
the time when a configuration and an angle of rotation are in agreement, and a Fourier descriptor 
is in agreement. Therefore, a Fourier descriptor and the power to each frequency can estimate 
how many configurations and appearance of two figures are in agreement. 

[0053] As mentioned above, the feature of an appearance can be expressed by the vector which 
consists of a Fourier coefficient and power, and the difference of an appearance can be 
expressed using the Euclidean distance between these feature vectors. Therefore, the degree 
HSh of different kind of the appearance of each field is computed by substituting the average of 
distance with the mean vector of all fields, and distance, and the standard deviation of distance 
for several 4. 

[0054] Next, the degree Hho of different kind of a hole is explained. In order to express the 
feature of the hole which a field includes, it is necessary to take into consideration not only the 
configuration of a hole but the number and hole site of a hole. Asking for the primary moment of 
a field can define the difference in a hole site. In addition, the primary moment is indicated by 
1992 "an image-analysis handbook." Therefore, the feature of a hole was expressed by the 
vector as shown in several 9. In addition, turn of a hole is made into the order near the zero of a 
picture. 
[0055] 
[Equation 9] 

ho=[nh, M, han, ■",ha lfc .hbLi,"",hb J „,'",ha«Ki, ••••ha,**. hh-o. hb. w ] 
fc«U 

ho : ft®»0HflK9i* 
nh ; : «iJ*i©ft©» 
Mi : fttti©*-^ 

ha, , , : flMt i ©1 8 1 ©ft© j*©7-'Ji«ft 

hb, u : ffltit i ©1 S 9 ©ft© j*©if 9- 

[0056] Therefore, the degree Hho of different kind of the hole of each field is computed by 
substituting the average of distance with the mean vector of all fields, and distance, and the 
standard deviation of distance for several 4. 

[0057] Moreover, the degree FP of [ ****** ] shown in several 2 can be defined as shown in 

several 10. 

[0058] 

[Equation 10] 
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